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ABSTRACT 

A mixed-method quasi-experimental study was designed to determine how 130 undergraduates in 
an introductory astronomy survey course laboratory changed their understanding of scientific 
inquiry working as individuals in relative isolation compared to working in small, collaborative 
learning groups when using specially designed astronomy curricula based on a backwards faded 
scaffolding approach (Slater, Slater, & Lyons, 2010). The results of this tw’o-group comparison 
study were determined by examining the change in responses from the pretest to the posttest 
administration of the Views of Scientific Inquiry (VOSI) survey (Schwartz, Lederman, & 
Lederman, 2008), the increase in scores on laboratory exercises, and analysis of field notes from 
the instructor. Analysis suggests that all students increased their understanding of scientific 
inquiry and astronomy when using the specially designed course materials regardless if the 
students were working in groups or individually in the learning laboratories. However, qualitative 
analysis yields further insight that there exists a qualitative difference in the complexity of 
research questions students working in groups elect to pursue over those students working 
independently. 
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Nature Of Science 


raknoi (2001) estimated that at least 250,000 undergraduate non-science majors enroll in an 
(Ay introductory astronomy survey course each year. Lor many of these students, this course satisfies a 
typical general education requirement for physical science course credits, and it is often the last 
science course they take in their college experience. The critical positioning of this course extends beyond 
increasing the scientific literacy of undergraduate non-science majors: An estimated 40% of introductory science 
course enrollees eventually become licensed teachers (Lawrenz, Huffman, & Appeldoorn, 2005). These future 
teachers’ knowledge and attitudes regarding science engendered during their basic college physical science courses 
could have a dramatic impact on generations of children when they are at their most impressionable (Lraknoi, 2001). 


A science course with a laboratory experience component serves as one of the long-standing general 
undergraduate education requirements common to many colleges and universities. Consistent with recommendations 
of the American Association for the Advancement of Science (1990, 1993), the National Research Council (2005), 
and the Society of College Science Teachers (1993), one of the objectives frequently included in the description of 
many of these courses is that a student will understand the nature and processes of scientific inquiry. Given that this 
intent represents such a universally called for course learning goal, it seems worthwhile to ascertain if particular 
aspects of the design and implementation of laboratory courses can explicitly meet this objective. 

In order to adequately measure students’ understanding of scientific inquiry, discipline-based astronomy 
education researchers first need to define inquiry, identify the important components worth studying, and then use 
appropriate measuring instruments. This necessity presents a challenge because a widely accepted consensus 
definition of inquiry has not yet been founded (NRC, 2005; Schwartz et ah, 2008). Harwood (2004) has argued that 
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the traditional linear scientific method, as it has been taught in schools for many years, is limited, and has even given 
an incorrect understanding of how contemporary science is actually done. To further illuminate how science is done, 
Schwartz (2004) examined how a number of cross-disciplinary scientists viewed scientific inquiry, reviewed 
instruments that were designed to measure an understanding of the nature of science (NOS) and the nature of 
scientific inquiry (NOSI), and eventually produced an instrument (2008) that compared the sophistication of 
students’ views of scientific inquiry (VOSI) compared to that of the scientists. 

The traditional astronomy laboratory exercises have long been criticized as being less than intellectually 
engaging by frequently asking students to graph an abstract set of provided data, such as the luminosity of stars 
versus their temperatures, and to confirm concepts discussed earlier in lecture or in their textbook. In an effort to 
increase the level of student engagement in inquiry, S. Slater led the development of a new set of astronomy 
laboratory learning activities that purposefully scaffold a student from guided inquiry activities to open inquiry ones 
and are based on a backwards faded scaffolding (BFS) design (Slater, Slater, & Shaner, 2008). Preliminary results 
show the success of these laboratories in supporting students’ understanding of the nature of scientific inquiry 
(Stewart, 2013; Lyons 2011). To examine more deeply the effectiveness of this BFS instructional design, it is 
valuable to know how students increase their understanding of inquiry individually compared to working in small 
learning teams. A more complete understanding adds to the growing astronomy education literature based on how to 
engage students in the “doing” of science. Moreover, a better understanding serves as a valuable precursor to 
offering astronomy laboratories online where off-campus, place-bound students would most likely be completing the 
laboratory learning experiences individually. 

In response to the needs described above, this study focused on a single research question: 

When using specifically designed inquiry-based laboratory activities that scaffold student learning to 
engage in designing and conducting open inquiry, how do students in an undergraduate astronomy 
laboratory increase their understanding of inquiry when working individually as compared to working in 
small, collaborative learning groups? 


METHOD 


Participants 

To answer this question, this study adopted a two-group, pretest/posttest quasi-experimental mixed-method 
design (Slater, Slater, & Bailey, 2011, p. 36). The sample of students studied was of local age-of-consent and 
volunteered to participate without compensation. Undergraduates were invited from all of those taking selected 
introductory astronomy laboratory classes one semester at a Western regional, doctoral-granting, research-extensive 
university. The demographic data of introductory astronomy students demonstrated a variety of ages and ethnicities, 
but most were non-minority traditional-aged undergraduates with ages ranging from 18-24. 

In this study, the majority of students completed their exercises in small, self-formed, collaborative 
laboratory groups that constituted one treatment group. A second treatment group was composed of students who 
completed their inquiry-based laboratory exercises individually. Students were not given a choice as to which 
treatment group they were assigned. 

Procedure 

The two-groups of students were given the Views of Scientific Inquiry (VOSI) survey (Schwartz, 
Lederman, & Lederman, 2008), in a pre- and posttest format. For the pretest administration, a total of 131 students 
responded, and 128 students responded to the posttest administration of the VOSI questionnaire. One out of eight 
laboratory classes had students complete their exercises individually to compare to those who completed their 
laboratories in groups of two, three, or four students. The class performing individually had 16 students the first 
week, but only 12 of these could be paired with the final week as opposed to 25 paired first week/last week scores 
for the group condition. Likely, some students from the groups did not attend the last week’s laboratory, but the 
criteria for inclusion required that at least two members must be in common from the first week to the last week to 
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be paired. Some of these students, therefore, could be missing, and the rest of the group would make up for them 
because only one score was given per group. 

All students in both groups shared the results of their inquiries with the larger class to receive feedback and 
to give feedback to others. These specially designed exercises tasked students to use and ask research questions of 
astronomical data available online through National Aeronautics and Space Administration (NASA) sources, 
Caltech's Jet Propulsion Laboratory (JPL), and other sources and are described extensively elsewhere (Slater, Slater, 
& Lyons, 2010; 2011). Each student or student-group needed computer access with a working Internet connection. 
The students working individually were not isolated from any peer-to-peer interaction. 

DATA AND RESULTS 

The version of Schwartz’s (2008) VOSI instrument used for this study targeted undergraduate, non-science 
majoring students enrolled in a liberal arts, general education course that satisfied a curricular distribution 
requirement for a laboratory-based science course. The student-supplied responses from the pre- and posttests of the 
VOSI were in the form of qualitative written narratives reviewed and coded using descriptors. These narratives were 
analyzed and placed into categories of level of understanding to see if a difference could be determined before and 
after the course. Some common elements in the responses were reported qualitatively in the form of a narrative. 

In addition, student artifacts from two different laboratory exercises were chosen for detailed analysis, one 
at the beginning of the course and one at the end of the course, and were considered functionally equivalent in terms 
of difficulty and mathematical sophistication needed. The open inquiry component of these exercises included a 
combination of a research question, data, and analysis by the students. This section was analyzed according to a 
rubric that focused on inquiry and described in detail elsewhere (Sibbernsen, 2010), and that rubric presented the 
results as numerical data. In turn, the data were used to analyze differences between the groups. 

The qualitative data and the quantitative data were compared to see if they presented similar results. The 
qualitative data and the quantitative data in this mixed method study were given equal status and were collected 
concurrently (Johnson & Onwuegbuzie, 2004). 

Qualitative - VOSI Data 

The VOSI questionnaire was administered as a pretest and a posttest to all of the introductory astronomy 
laboratory students enrolled, but only the data from volunteering students are included in this analysis, which is 
effectively 100%. These questionnaires were analyzed to determine how students’ views of scientific inquiry 
changed over the course of the semester. For the pretest administration, a total of 131 students responded, and 128 
students responded to the posttest administration of the VOSI questionnaire. One out of eight laboratory classes had 
students complete their exercises individually to compare to those who completed their laboratories in groups. The 
VOSI pretests and posttests were paired for all students in this class to examine closely their changes in views of 
scientific inquiry to see if they corresponded to the entire surveyed group. There were 15 VOSI pretests and 
posttests paired for the individual completion laboratory group. Therefore, 116 students were in the group 
completion section for the pretest VOSI and 113 for the posttest. Each of the seven questions of the VOSI 
questionnaire corresponds to a different component of the understanding of inquiry and the nature of how scientists 
do science. The results of each of these questions are addressed separately, as prescribed by Schwartz, Lederman & 
Lederman (2008). 

VOSI Question 1 

This first question examined views of activities that scientists do when they “do science.” Specifically, 
students were asked, “What types of activities do scientists (e.g., biologists, chemists, physicists, earth scientists) do 
to learn about the natural world? Discuss how scientists (biologists, chemists, earth scientists) do their work. ” 
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Table 1: Number Of Responses To Question 1 


Group Completion 

Naive 

n(% of Total) 

Too General 
n(% of Total) 

Typical Response 

Informed View 

Pretest 

15 (13%) 

44 (38%) 

51 (44%) 

7 (6%) 

Posttest 

1 (1%) 

25 (22%) 

81 (72%) 

6 (5%) 

Individual completion 

Pretest 

3 (20%) 

4 (27%) 

6 (40%) 

2 (13%) 

Posttest 

0 (0%) 

3 (20%) 

11 (73%) 

1 (7%) 

Frequency of Terms or Phrases Used 


Pretest 

Posttest 


Experiment 


76 

83 


Ask questions 


15 

42 


Make hypothesis 


41 

35 


Collect/analyze data 


55 

52 


Make observations 


56 

51 


Scientific method 


24 

11 


Test theories 


26 

11 


Make conclusions 


12 

25 



An example of a naive response would be, “They do their work by discovering new things by going 
outside.” An example of a response that would be too general would be, “Scientists run tests to answer questions.” A 
typical response for this question includes a description of activities such as, “observe, experiment, ask questions, 
make hypotheses, collect and analyze data, and draw conclusions.” An informed view includes all of the typical 
responses and also includes such activities as, “collaborate with other scientists and publish in peer-reviewed 
journals.” 

VOSI Question 2 

This second question explores the understanding of the scientific distinction between the terms “data” and 
“evidence.” In part (a) of the question, students were asked, “What does the word “data” mean in science?” 


Table 2: Number Of Responses To Question 2 (a) 


Group Completion 

Narrow Response 
n(% of Total) 

Typical Response 
n(% of Total) 

Pretest 

16 (14%) 

100 (86%) 

Posttest 

13 (12%) 

100 (88%) 

Individual completion 

Pretest 

2 (13%) 

13 (87%) 

Posttest 

0 (0%) 

15 (100%) 


For part (a), a narrow response might suggest that data must be numerical such as, “the numbers collected 
in an experiment.” A typical response for part (a) would be, “Data is information gathered from an experiment 
through observation and can be quantitative or qualitative.” 

Students were asked in part (b) of this question, “Is ‘data’ the same or different from ‘evidence’?” 


Table 3: Number Of Responses To Question 2 (b) 


Group Completion 

Same 

Different 

Similar/Could Be/Not Sure 

n(% of Total) 

n(% of Total) 

n(% of Total) 

Pretest 

20 (17%) 

77 (66%) 

19 (16%) 

Posttest 

25 (22%) 

79 (70%) 

9 (8%) 

Individual completion 

Pretest 

3 (20%) 

11 (73%) 

1 (7%) 

Posttest 

7 (47%) 

8 (53%) 

0 (0%) 
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Question three examines what the students view as an experiment. The question specifically asked, “A 
person interested in animals looked at hundreds of different types of animals who eat either meat or plants. He 
noticed that those animals who eat similar types of food tend to have similar teeth structures. For example, he 
noticed that meat eaters, such as lions and coyotes, tend to have teeth that are sharp and jagged. They have large 
canines and large, sharp molars. He also noticed that plant eaters, such as deer and horses, have smaller or no 
canines and broad, lumpy molars. He concluded that there is a relationship between teeth structure and food source 
in the animals, (a) Do you consider this person’s investigation to be an experiment?” 


Table 4: Number Of Responses To Question 3(a) 



Group Completion 

Yes 

n(% of Total) 

No 

n(% of Total) 

No Answer 
n(% of Total) 


Pretest 

Posttest 

36 (31%) 

52 (46%) 

80 (69%) 

59 (52%) 

0 (0%) 

2 (2%) 


Individual completion 

Pretest 

Posttest 

7 (47%) 

3 (20%) 

8 (53%) 

12 (80%) 

0 (0%) 

0 (0%) 


Students were asked in 
scientific? Please explain why or 

part (b) of this question, “ 
why not by describing what it 

'Do you consider this person’s investigation to 
means to do something ‘scientifically.’” 

be 


Table 5: Number Of Responses To Question 3(b) 



Group Completion 

Yes 

n(% of Total) 

No 

n(% of Total) 

No answer 
n(% of Total) 


Pretest 

Posttest 

82 (71%) 

68 (60%) 

34 (29%) 

45 (40%) 

0 (0%) 

1 (1%) 


Individual completion 

Pretest 

Posttest 

8 (53%) 

11 (73%) 

7 (47%) 

4 (27%) 

0 (0%) 

0 (0%) 



The study described here is not a conventional experiment in that it does not manipulate a single variable 
while keeping all the others the same, so the desired response for part (a) shown in table 4 is “No.” This constitutes 
an observational study. Part (b) examines whether students regard this as a scientific study, and the desired response 
shown in table 5 is “Yes.” A respondent with a more narrow view of how science is done would be more likely to 
say that it is not scientific because only one way to do the scientific method exists, and that requires an experiment 
(Sibbernsen, 2010). 

VOSI Question 4 

This question directly examines the viewpoint of “the scientific method” and follows directly from question 
three. If responding students specify that the only way to do science is through the scientific method, they describe 
the method of asking questions, performing an experiment, analyzing information gathered, and coming to a 
conclusion. The more inclusive description of the scientific method suggests that good science can be conducted in 
many ways and includes observational and descriptive studies. The students were specifically asked, “The ‘scientific 
method’ is often described as involving the steps of making a hypothesis, identifying variables 
(dependent/independent), designing an experiment, collecting data, reporting results. Do you agree that to do good 
science, scientists must follow the scientific method?” 


Table 6: Number Of Responses To Question 4 


Group Completion 

Yes 

n (% of Total) 

No 

n(% of Total) 

No Answer 
n(% of Total) 

Pretest 

83 (72%) 

33 (28%) 

0 (0%) 

Posttest 

82 (73%) 

30 (27%) 

1 (1%) 

Individual completion 

Pretest 

13 (87%) 

1 (7%) 

1 (7%) 

Posttest 

14 (93%) 

1 (7%) 

0 (0%) 
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This question examines the respondents’ views of how scientists justify their findings. This question 
specifically asked, “What type of information is required for scientists to justify and accept a scientific claim? (In 
other words, how do scientists know when they are ready to make their research results public? What do they need 
in order to convince others of their claim?)” 


Table 7: Number Of Responses To Question 5 


Group Completion 

Naive 

n(% of Total) 

Typical/informed responses 
n(% of Total) 

Other/not sure/blank 
n(% of Total) 

Pretest 

60 (52%) 

54 (47%) 

2 (2%) 

Posttest 

48 (42%) 

64 (57%) 

1 (1%) 

Individual completion 

Pretest 

7 (47%) 

7 (47%) 

1 (7%) 

Posttest 

4 (27%) 

11 (73%) 

0 (0%) 

Frequency of Terms or Phrases Used 


Pretest 

Posttest 

Repeatable 


51 

61 

Data/statistics 


38 

48 

Evidence/proof 


37 

58 

Publish/peer review 


37 

43 

Reliability/validity 


25 

10 


An example of a naive response to this question would be, “They would need good, solid data in order to 
prove or disprove their theory.” A typical response would include discussions of reliability, validity, evidence, 
proof, if the data are consistent, repeatable, and if the assertion is falsifiable. Informed responses might include all of 
these and discuss the need for peer review, collaborative work with other scientists, and publishing or presenting 
their work at conventions. 

VOSI Question 6 

This question probes the understanding of interpretation in data analysis to reach conclusions. Students 
were asked in part (a), “If several scientists, working independently, ask the same question and follow the same 
procedures to collect data, will they necessarily come to the same conclusions?” 


Table 8: Number Of Responses To Question 6(a) 


Group Completion 

Same 

n(% of Total) 

Different 
n(% of Total) 

Maybe/Depends/Blank 
n(% of Total) 

Pretest 

15 (13%) 

93 (80%) 

8 (7%) 

Posttest 

28 (25%) 

78 (69%) 

7 (6%) 

Individual completion 

Pretest 

2 (13%) 

12 (80%) 

1 (7%) 

Posttest 

5 (33%) 

10 (67%) 

0 (0%) 


A more nuanced understanding would suggest that some subjectivity must be taken into account in the 
analysis of similar data while still producing valid conclusions. A naive response would state that they should get the 
same results unless one of them made a mistake in his/her analysis. Some respondents replied that they should get 
the same answers because the underlying physical processes should be the same regardless of the procedure, and 
others responded that they should get the same results because they are analyzing the same or similar data and 
suggests that there is no subjectivity in the interpretation. 

For part (b) of this question, students were asked, “If several scientists, working independently, ask the 
same question and follow different procedures to collect data, will they necessarily come to the same 
conclusions?” 
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Table 9: Number Of Responses To Question 6(b) 

Group completion 

Same 

n (% of Total) 

Different 
n(% of Total) 

Maybe/Depends/Blank 
n (% of Total) 

Pretest 

4 (3%) 

88 (76%) 

24 (21%) 

Posttest 

8 (7%) 

95 (84%) 

9 (8%) 

Individual completion 

Pretest 

0 (0%) 

12 (75%) 

3 (20%) 

Posttest 

0 (0%) 

14 (93%) 

1 (7%) 

Parts (c) and (d) ask if the answers to (a) and (b) should be the same 

or different if the scientists work 

together. In general, they could possibly get different results, but it is more likely that a consensus will be reached. 

Students were asked in part (c), 

“Does your response to (a) change if the scientists are working together?” 


Table 10: Number Of Responses To Question 6(c) 


Group completion 

Same 

n(% of Total) 

Different 
n(% of Total) 

Maybe/depends/blank 
n(% of Total) 

Pretest 

43 (37%) 

55 (47%) 

18 (16%) 

Posttest 

53 (47%) 

54 (48%) 

6 (5%) 

Individual completion 

Pretest 

7 (47%) 

6 (40%) 

2 (13%) 

Posttest 

6 (40%) 

8 (53%) 

1 (7%) 

Students were asked in part (d), 

“Does your response to (b) change if the scientists 

are working together?” 


Table 11: Number Of Responses To Question 6(d) 


Group completion 

Same 

n(% of Total) 

Different 
n(% of Total) 

Maybe/depends/blank 
n(% of Total) 

Pretest 

27 (23%) 

73 (64%) 

16 (13%) 

Posttest 

24 (17%) 

79 (70%) 

15 (13%) 

Individual completion 

Pretest 

3 (20%) 

11 (73%) 

1 (7%) 

Posttest 

3 (20%) 

11 (73%) 

1 (7%) 


VOSI Question 7 

The last question examines how respondents view outliers or inconsistencies in the data. Students were 
specifically asked, “Occasionally, scientists encounter inconsistencies in their data. What do you think scientists do 
when some part of their data does not fit with what they expect (an ‘outlier’ or inconsistency is found)?” 


Table 12: Number Of Responses To Question 7 


Group completion 

Naive 

n(% of Total) 

Typical response 
n(% of Total) 

Informed view 
n (% of Total) 

Blank 

n (% of Total) 

Pretest 

20 (17%) 

83 (72%) 

10 (7%) 

1 (1%) 

Posttest 

15 (13%) 

87 (77%) 

8 (7%) 

3 (3%) 

Individual completion 

Pretest 

2 (13%) 

12 (80%) 

0 (0%) 

1 (7%) 

Posttest 

1 (7%) 

13 (87%) 

1 (7%) 

0 (0%) 


Frequency of Terms or Phrases Used 

Pretest 

Posttest 

Redo/try again 

46 

53 

Fix/find what is wrong 

36 

38 

Revise/adjust/different tests 

16 

24 

Throw it out/disregard 

19 

30 

New discoveries 

9 

8 


An example of a naive response would be, “They don’t take it into account and record it as a miss done 
[sic] experiment using no data from it.” A typical response is that, “They should repeat the experiment to find out 
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where the mistake is made.” Even though this is the typical response, thinking of anomalies as being caused by a 
mistake in the experiment lacks desirability. An informed view might suggest that the anomalies could lead to new 
experiments and new discoveries. 

Quantitative - Laboratory Exercise Data 

A non-authoring college astronomy professor from another campus who was not teaching the participants 
study classes volunteered to do the qualitative analysis without knowing the individual students in the study. This 
expert who performed the grading was deeply familiar with the content and was qualified to assess the quality of the 
laboratory exercises performed the grading. Only the open inquiry phases of laboratory exercises were evaluated 
because this was the final phase and a culmination of all of the steps: the development of a research question, 
producing a procedure, taking the data, and presenting a conclusion. 

A two-group unpaired f-test was applied to the first week laboratory exercise scores for the individual 
condition and the collaborative learning group condition to see if a statistical difference could be identified between 
them. Likewise, a two-group unpaired f-test was applied to the last week laboratory exercises scores for the two 
conditions. The mean scores for the two conditions differ significantly for the first week scores and for the final 
week scores. These f-tests are deemed appropriate because each of the four data sets satisfies the assumptions of 
normality for each as shown by the Kolmorgorov-Smirnov (Stephens, 1974) comparison of data sets, and each of the 
pairs of data sets satisfies the assumption of equality of variances as shown by the Bartlett Test (Snedecor & 
Cochran, 1989). 


Table 13: Laboratory Exercise Scores Two-Group Unpaired f-tests 


Statistics 

Individuals 
First Week 

Groups 

First Week 

Individuals 

Last Week 

Groups 

Last Week 

Mean 

21.42 

19.72 

22.50 

24.28 

Standard Dev. 

3.78 

3.37 

2.11 

2.70 

f-value 

Two-tailed p-value 
f-test decision (p < .05) 

Difference is 

1.3784 

0.1768 

not statistically significant 

2.0033 

0.0529 

Difference is not statistically significant 


A paired f-test was performed for the group exercises to see if a significant increase became apparent in the 
scores from the first week to the last week of laboratory exercises. 


Degrees of Freedom 

f-value 

Two-tailed p -value 
f-test decision (p < 0.05) 


Table 14: Laboratory Exercise Group Scores Paired f-test 
' 24 

5.9862 

< 0.0001 

Statistically Significant 


DISCUSSION 


Qualitative - VOSI 

The VOSI allows evaluation of several components of the understanding of scientific inquiry. In general, 
the results indicate that students’ understanding of scientific inquiry increased for some of the questions and 
remained unchanged for others. More specifically, for the questions that correspond to activities in the focus of the 
astronomy laboratory, the students appear more comfortable and more adept with the language of science and 
inquiry, and generally show a greater and nuanced understanding of those processes. 
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Table 15: Summary Of VOSI Results 


VOSI 

Groups 

Individuals 

Question 1 - How do scientists do their work? 

Increase 

Increase 

Question 2 - Data vs. Evidence 

Slight Increase 

Slight Increase 

Question 3 - Is a process scientific or not? 

Slight Decrease 

Slight Increase 

Question 4 - Follow the scientific method? 

No Change 

No Change 

Question 5 - Scientists justifying claims 

Increase 

Increase 

Question 6 - Conclusions working independently 

Slight Increase 

Slight Increase 

Question 7 - Outliers 

Slight Increase 

Slight Increase 


For both the group and individual conditions, there occurred a large increase in the percentage of students 
who understood how scientists do science (question 1), a moderate increase in the percentage of students who 
understood how scientists justify and accept their claims (question 5), and a slight increase in the percentage of 
students who increased their knowledge of how scientists deal with outlying data points (question 7). For all of these 
increases, students working individually did as well as the students working in self-formed, small, collaborative 
learning groups in terms of giving typical or informed responses rather than naive ones. These three questions acted, 
arguably, as the most important in terms of understanding scientific inquiry, and were the primary focus of the 
laboratory exercises. 

Very little change occurred (questions 2, 4, and 6), or mixed trends became identifiable (question 3) 
between the two conditions in the remaining questions. For example, in question two, part (a), only a small portion 
(about 13%) of respondents gave a narrow response to the question of what “data” means in science, and this 
percentage remained relatively unchanged in the posttest for both conditions. This percentage can be interpreted as 
unexpectedly high in relation to respondents who understood the difference before the course began. 

Qualitative - Independent Grader and Instructor Interview 

Although the first and the last week laboratory exercises focused on different core astronomy concepts, the 
independent expert evaluator judged the quality of inquiry in the last week as better than the first week. The 
instructor judged that the students grew in their ability to think critically. The VOSI analysis supports this 
perception. 

One particularly interesting feature of the instructor’s field notes stands out above all others. Students 
working as individuals, as compared to their counterparts who were working in groups when completing the BFS 
laboratory exercises, wrote more than the groups, took longer to do the labs, and designed conservative research 
questions that were “safer.” The instructor generalized that three types of groups tend to develop: equal participation 
among all members, a group of three with one student quieter but still participating, and a group of three with one 
student quieter and not participating as much. In the follow-up questions with the instructor, it was suggested that 
the groups had an advantage over the individuals because of the division of labor, and that if there was no time 
constraint on the lab, this may not have as much of an effect. No more details are available on this aspect as it was 
not an anticipated result, and future studies could benefit if this became a focus of further investigation. 

Quantitative - Graded Laboratory Exercises 

Laboratory exercises were collected the first week and the last week to determine the probability of an 
increase in the scores as determined by an inquiry-based rubric (Sibbernsen, 2010). The first week to last week 
paired exercise f-test scores determined that the difference in averages for the group condition was strongly 
statistically significant with a /;-value less than 0.0001. This indicates that they performed better on inquiry during 
the last week as opposed to the first. 

The individuals’ average scores also increased from the first to the last week. As opposed to 25 groups with 
paired scores from the first to the last week, the individual scores could only be paired for 12 of the individuals. The 
individual condition had a higher mean score in the first week, 21.42, compared to 19.72 for the group condition. 
From classroom observations, this may be because the individuals spent more time and took more care with writing 
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their laboratory exercises at the beginning. The group condition had a higher mean for the last week exercises, 24.28 
compared to 22.50, but these did not differ statistically from each other. 

CONCLUSIONS 

The VOS1 survey functioned as an effective tool for analyzing and understanding students’ developing 
sense of scientific inquiry. Given that the participating students were largely Caucasian and of traditional college 
student age, a fruitful future study could be expanded to include different populations using this instrument as a 
pretest/posttest and could include observations made throughout the semester. The composition of the sample is a 
potential limitation for generalizing the results of this study across diverse populations. Accordingly, further testing 
varying the sample and expanding the analysis is recommended. The results from this study that used four-year 
university students could be compared to a sample of two-year community college students. In addition, the 
demographics were not considered in this study. Further studies could examine differences between male and female 
students, different ages, different learning preferences for working individually or in groups, and racial and ethnic 
differences. 

An additional avenue for future study is to perform the pre/post test VOSI for lecture-only astronomy 
classes to isolate the potentiality of this astronomy laboratory being solely responsible for the increase in 
understanding of scientific inquiry. A further extension of this work would be to administer the VOSI to students 
performing the inquiry-based BFS astronomy exercises in an online laboratory situation. Even if students in this 
situation were allowed to select whether they do the exercises individually or in groups, it is anticipated that most of 
these students will choose to perform the experiments individually given the challenges of group work in the online 
environment. 

The success of the BFS technique for teaching inquiry has been demonstrated and should translate to 
sciences other than astronomy, such as geology or physics. Per recommendation, workshops should be held for 
interested instructors to obtain training on the technique to develop new backwards faded scaffolding inquiry-based 
laboratory exercises and to learn how to test and analyze students’ increase in understanding of inquiry. From these 
workshops, networks of faculty teaching-participants could be built to share produced materials. 
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